Skip to content

updating loading in stable lm demo to use transformer bridge#1023

Merged
jlarson4 merged 36 commits intodev-3.x-canaryfrom
stable_lm_demo_transformer_bridge_migration
Apr 3, 2026
Merged

updating loading in stable lm demo to use transformer bridge#1023
jlarson4 merged 36 commits intodev-3.x-canaryfrom
stable_lm_demo_transformer_bridge_migration

Conversation

@degenfabian
Copy link
Copy Markdown
Collaborator

@degenfabian degenfabian commented Aug 19, 2025

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes # (issue)

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Screenshots

Please attach before and after screenshots of the change if applicable.

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

@bryce13950 bryce13950 added this to the 3.0 milestone Aug 20, 2025
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x' into stable_lm_demo_tra…
f44faa6 
…nsformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x' into stable_lm_demo_tra…
a99dfb6 
…nsformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x' into stable_lm_demo_tra…
06c0732 
…nsformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x' into stable_lm_demo_tra…
ebc1feb 
…nsformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x' into stable_lm_demo_tra…
1a0f780 
…nsformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x' into stable_lm_demo_tra…
2fdd6e0 
…nsformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x' into stable_lm_demo_tra…
59f7fe2 
…nsformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x' into stable_lm_demo_tra…
e53db49 
…nsformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x' into stable_lm_demo_tra…
69f9e7c 
…nsformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x' into stable_lm_demo_tra…
e8516b4 
…nsformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x' into stable_lm_demo_tra…
e0914a1 
…nsformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x' into stable_lm_demo_tra…
b069985 
…nsformer_bridge_migration
@bryce13950 bryce13950 changed the base branch from dev-3.x to dev-3.x-folding October 10, 2025 13:06
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x-folding' into stable_lm_…
1904dce 
…demo_transformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x-folding' into stable_lm_…
17f27e8 
…demo_transformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x-folding' into stable_lm_…
d1d7af3 
…demo_transformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x-folding' into stable_lm_…
49a1695 
…demo_transformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x-folding' into stable_lm_…
4854ab6 
…demo_transformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x-folding' into stable_lm_…
6a3de9f 
…demo_transformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x-folding' into stable_lm_…
18c7177 
…demo_transformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x-folding' into stable_lm_…
de0f324 
…demo_transformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x-folding' into stable_lm_…
bf93d3b 
…demo_transformer_bridge_migration
@bryce13950
fixed booting
2be7624
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x-folding' into stable_lm_…
7057314 
…demo_transformer_bridge_migration
@bryce13950
added dtype selction
15a1e48
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x-folding' into stable_lm_…
824e3d3 
…demo_transformer_bridge_migration
@bryce13950
Merge remote-tracking branch 'origin/dev-3.x-folding' into stable_lm_…
a2fc9a5 
…demo_transformer_bridge_migration
@jlarson4 jlarson4 changed the base branch from dev-3.x-folding to dev-3.x February 9, 2026 16:25
@jlarson4 jlarson4 self-assigned this Apr 3, 2026
@jlarson4 jlarson4 changed the base branch from dev-3.x to dev-3.x-canary April 3, 2026 17:29
@jlarson4
Copy link
Copy Markdown
Collaborator

jlarson4 commented Apr 3, 2026

I ended up having to rewrite a significant portion of this notebook. I was unable to recreate the saved data on this version or on 2.x's Hooked Transformer, I attempted to recreate the output on main or dev. As far as I am aware, this notebook became inaccurate long ago due to a bug in abstract attention.

In July 2023 when this demo was created in #354: components.py had the IGNORE buffer set to -1e5 (the bug). This is what produced the results that were demonstrated in the original version of the notebook.

In September 2023: IGNORE was changed to -torch.inf in addition to zeroing out NaNs caused by applying softmax to a vector of -infs. This occurred across PR #366 & PR #386.

The current behavior appears to be the correct behavior, and the original details of this demo were a side effect of the -1e5 buffer bug.

@jlarson4 jlarson4 merged commit b00a517 into dev-3.x-canary Apr 3, 2026
19 checks passed
@jlarson4
Copy link
Copy Markdown
Collaborator

jlarson4 commented Apr 3, 2026

Additionally, I left this commented out of CI for now. The StableLM Alphas are too large to run within the constraints of our current CI infrastructure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@jlarson4 jlarson4

Labels

None yet

Projects

None yet

Milestone

3.0

Development

Successfully merging this pull request may close these issues.

3 participants

@degenfabian @jlarson4 @bryce13950